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Abstract 

In this paper we prove some results on interior transmission eigenvalues. First, under rea- 
sonable assumptions, we prove that the spectrum is a discrete countable set and the generalized 
eigenfunctions spanned a dense space in the range of resolvent. This is a consequence of spectral 
theory of Hilbert- Schmidt operators. The main ingredient is to prove a smoothing property of 
resolvent. This allows to prove that a power of the resolvent is Hilbert-Schmidt. We obtain an 
estimate of the number of eigenvalues, counting with multiplicities, with modulus less than 
when t is large. We prove also some estimate on the resolvent near the real axe when the square 
of the index of refraction is not real. Under some assumptions we obtain lower bound on the 
resolvent using the results obtained by Dencker, Sjostrand and Zworski on the pseudospectra. 
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1 Introduction 



In this paper we prove existence of infinite number of interior transmission eigenvalues under some 
condition on the index of refraction. Remind the problem. Let a smooth bounded domain in M". 
Let n{x) a smooth function defined in Q, called the index of refraction. The problem is to find k and 
(ui, v) such that 

Aw + k'^n[x)w = in ri, 

Av + fc2„ = in fi, ,^ 
w = V on 9ri, 
d^w ~ dyV on (9i7, 

where 9^ is the exterior normal derivative to dO,. We consider here the function n{x) complex valued. 
In the physical models, we have n{x) = ni (x) + m2 (x) / k where rij are real valued. Taking u = w — v 
and V = k'^v, we obtain the following equivalent system 

(A + fc2(l + m)) u + mv = in il, 

(A + k^)v = in n, (2) 
u = d,^u = on do,, 

where, for simplicity, we have replaced v hy v and n by 1 + m. 

Under some assumptions on for instance our results apply for n{x) = ni{x) + in2{x)/k 

where rij are real valued, we prove that the associated resolvent is compact on H'^{Vl) L'^{VL) (see 
Theorem 2) and we have a countable set of fc| and generalized finite dimensional eigenspace Ej 
such that Uj-gN-Ej spanned a dense space in the range of the resolvent (see Theorem 3). When n{x) 
is real, Paivarinta ans Sylvester [22] have proved that there exist interior transmission eigenvalues, 
Cakoni, Gintides and Haddar [5] proved that the set of fcj is infinite and discrete. For n{x) complex 
valued Sylvester [23] proved that this set is discrete finite or infinite. The case where nix) = 1 in 
a part of Vl that is the presence of cavities in fi was considered by Cakoni, (^ayoren and Colton ]3], 
Cakoni, Colton and Haddar ]4]. Here maybe because we use pseudo-differential calculus we do not 
have problems with cavities. In ]13, 14] Hitrik, Krupchyk, Ola and Paivarinta studied same type of 
problems where the Laplacian is replaced by an elliptic operator with constant coefficients of order 
m > 2. They proved in some cases, existence of interior transmission eigenvalue and the generalized 
eigenfunctions span a dense space. The proof uses the property of trace class operators and requires 
that m > n. Here as we consider the power of resolvent we have not this restriction and we consider 
the Laplacian for seek of simplicity but we can replace the Laplacian by a general elliptic operator 
of order 2 with real coefficients. 

We can complete this result giving a weak version of Weyl law. If we denote by N{t) the number 
of jfcjl, counting with multiplicities, smaller than t, we prove that N{t) < Ct^^'^ (see Theorem 4). In 
]f6], ]17] and ]18]; Lakshtanov and Vainberg study a problem as the (1) where the boundary condition 
d^w — d^v is replaced by d^w ~ a{x)d„v where a{x) ^ I for all x S 9f2. In this case the problem is 
elliptic in the sense that if (/, 5) G L^(f2) then (w, v) £ H^il) (see (5) for the definition of the system 
with force terms). In ]19] they prove an lower bound on the counting function N{t), if n(x) is real 
and for the problem (f ). To be precise they prove N{t) > Ct^ where C > 0. Actually they consider 
only the real eigenvalues but it is not clear that the bound is sharp even for real eigenvalues. 

In [8] Colton and Kress prove that if Imn(x) > for a\\ x £ ft and Imn 7^ then k^ is not real. 
Here we give an estimate on the resolvent for fc^ S M (see Theorem 5). This result is based on the 
Carleman estimates and following the same way as in the context of control theory, stabilization, 
scattering (see for instance [20], 121], ]10]). 

In some case (see Theorem 6) we can give lower bound on the resolvent using the result obtained 
by Dencker, Sjostrand and Zworski [12] on the pseudospectra. Even if the bounds obtained by 
Carleman's method and by the pseudospectra results have the same size, we cannot apply both 
methods in the same situation. 

The interest of the problem (1) is related with the Theorem 8.9 in Colton and Kress [8] first 
proved by Colton, Kirsch and Paivarinta ]7]. Here we give a quick survey of this result. Let n{x) 
defined in fl as the one in (1) and by 1 in M" \ il. We assume n{x) 7^ 1 in il. For fc e M, let u the 
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solution of the following problem 



Au + k'^n{x)u = in M", 

u ^ + u^, 

\imr^+oo r("-i)/2(^ - iku') = 0, 

where is a solution to Au* + fc^w* = in M", and r = We have u'^{x) = |^|'^„li')/2 '^oo(3^) + 
O ( i^ic^+D/a ) , where x = x/|x|. 

If u^{x) = e*'^^'*, where d G §"^^ we denote the corresponding Uoo by Uoo(i, d). If we assume that 
is connected and contains 0, the space spanned by Uoo{x,d) for d G E>"'~^ is dense in L^(§"~^) if 
and only if the space of solution of (1) is reduced to 0. This problem can be interpreted as follow, 
is a incident plane wave and Woo is the first relevant term of u created by the perturbation n localized 
in n. This kind of result is interesting in the field of inverse scattering problem. I let the reader 
interested by this field to find more information in [8] and in the recent survey given by Cakoni and 
Haddar [6]. 

1.1 Results 

Let ^l a "t^"" bounded domain in M". Let n{x) e complex valued. We denote by ra{x) — 

n{x) — 1. We consider also the case where n{x) = niix) + in2{x)/k where rij are real valued. We 
assume that for all a; € O, n{x) ^ 0, or ni{x) ^ or equivalently m(a;) ^ —1. We assume there exists 
a neighborhood W of dO. such that for x G n{x) 7^ 1 or ni{x) 7^ 1 or equivalently m{x) ^ 0. 
Actually if n(x) 7^ 1 for all x € 91), such a neighborhood W exists. 
Let Ce the cone in C defined by 

Ce = e C, 3x e II, 3A > 0, such that z = -A(l + m(x))}. (3) 

In the case where n = ni + m2/fc, Cg = (—00, 0] if nii^x) > —1 for all x Cz and Ce = [0, +00) 
if ni{x) < — 1 for all a; G il. 

Our regularity result will be given in the Sobolev spaces. We use the following notations. The 
i^(n)-norm will be denoted by || • ||. For s G K, we denote the usual H" norm in by HwUl^s = 
/(I + l?P)'*l^(C)P<^C where ii is the classical Fourier transform. The H'^ space on will be denoted 
by H {n) and we say that w a distribution in is in H (£7) if there exists (3 G such that 
= w. The norm is given by — inf{||/3||//s, where /3|f2 — w}. For q G N it is classical 

that is equivalent to J2\a\<q W^^xMl^ (see for instance [15, Vol. 3, Corollary B.2.5]). 

We denote by B^iu, v) = (/, g) the mapping defined from Hg{Q) (S {v e L^(ri), Ai; G ^^(il)} to 
L'^{n)®L^{n) by 

f i-r^A- z)u+ -rr-v f inn 

J \ 1+m / 1+m (A\ 

\ (A - z)v ^ g in n 

Theorem 1. Assume Ce 7^ C, then there exists z G C such that is bijective from HqITI) © {w G 

L^in), AveL^in)} to L^{n)® L^{n). 

If for z G C the solution Bz{u,v) — {f,g) exists we denote by Rzif,g) — {u,v). 

Theorem 2. Assume Ce 7^ C, there exists z G C such that the resolvent Rz from H (fl) © L'^(fl) to 
itself is compact. 

In particular, we can apply the Riesz theory, the spectrum is finite or a discrete countable set. If 
X ^ is in the spectrum, X is a eigenvalue with finite dimensional associated generalized eigenspace. 

Remark 1. This result improves the Sylvester's theorem [23, theorem 2[ with respect the geometrical 
assumption on to. Nevertheless, here the regularity assumption on m is stronger than the assumption 
given in [23[. 
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Remark 2. Actually if zq ^ Ce U (— oo,0] for all A large enough we can take z = Xzq in the 
Theorems 1 and 2. 

Remark 3. Actually we can also consider on L^{n) © L^(17) to itself. The range is in 11^(^1) © 
L^(ri). Then with our regularity results we can prove that i?^ is a- mapping from L^(ri) © L'^{U) to 
H (n) © H (ft). In particular i?^ is compact from i^(il) © to itself and we can deduce the 

same properties on the spectrum of as in Theorem 2. 

In general for a non self-adjoint problem, we cannot claim that the spectrum is non empty. In the 
following theorem, with a stronger assumption on Cg, we can prove that the spectrum is non empty. 

We say that Ce is contained in a sector with angle less than 9 if there exist 9i < 62, such that 
Ce C {z e C, z = or |fy = e''^, where di < ip < 6*2}, and 02 - Oi < 0. 

Theorem 3. Assume that Ce is contained in a sector with angle less than 9 with 9 < 2tt/p and 
< tt/2 where Ap > n. Then there exists z such that the spectrum of R^ is infinite and the space 
spanned by the generalized eigenspaces is dense in HQ{n) © {w S L^(r2), Av € L^(ri)}. 

Remark 4. This result is based on the theory given in Agmon [1] and using the spectral results 
on Hilbert-Schmidt operators. In this theory we deduce that the spectrum is infinite from the proof 
that the generalized eigenspaces is dense in the closure of the range of R^. Here we prove that Rp 
is a Hilbert-Schmidt operator if Ap > n. We can deduce the spectral decomposition of Rz from the 
one of RP . 

We can prove a weak Weyl law. Let Zj the elements of the spectrum of Rz and Ej the generalized 
associated eigenspace. We denote by N{t) — |-i<t2 dimEj. 

Theorem 4. Under the same assumption as in Theorem 3, then there exists C > such that 
N{t) < Ct^+". 

Remark 5. I do not know if this result is optimal. The estimate is lower than the usual Weyl law 
which is in This is due to the estimates obtained on the resolvent which are different than the 
one used to prove the usual Weyl law. 

Here we give some ideas to obtain the Theorems 3 and 4 using the method given in Agmon [1]. 
First we prove a regularity result, that is we consider the iterate of Rz, we have R^ is bounded 
from H'^ © L2 to ij2fc+2 ^ jj2k rpj^j^ implies that i?^ is an Hilbert-Schmidt operator if k is large 
enough and we can use the spectral theory for this operator class. The main problem to prove the 
regularity result is at the boundary. To do this we reduce the problem to the boundary by using 
the pseudo-differential calculus. It is well-known that for an elliptic problem, we can find a relation 
between the two traces of a solution. These two relations, for u and v in (4) and the assumption on 
the two traces of u allows to compute the trace of v by the data. Actually the coupling is very weak 
because it involves a lower order term, consequently we obtain a weak estimate. 

In the context of stabilization or control for wave equation, there are a lot of results on decreasing 
of energy obtained by Carleman estimates (see for instance [20, 21, 10]). This method allows to 
give quantitative results related with uniqueness result. We use here the same method to prove an 
estimate on the resolvent near the real axe for a complex index of refraction. The theorem below is 
an quantitative version of the Theorem 8.12 given by Colton and Kress [8]. Here it is more convenient 
to use the variables introduce in (1). Let {w,v) solutions of 



where {f,g) G L^(r2) © L'^{n). We denote Rk'^{f,g) = {w,v). Remark that Ri^2 exists except for 
a discrete set of k^. Indeed we can check that R_}^2[f — g, k^g) = {w — v, k'^v) this gives {w, v) if 
(— fc^)^^ is not in the spectrum of Rq. 



Aw + k'^n{x)w = / in Q, 
Av -\- k^v = g in O 
w ^ V on dfl 
di,w ~ d^v on dVl, 



(5) 
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Theorem 5. We assume that Imn > and Imn ^ or if n{x) = ni{x) + in2{x)/k, n2{x) > and 
n2 ^ 0. Then there exist Ci > and C2 > such that WRk^W < de'^^l'^l for all k € R. Here \\ ■ \\ 
denote the norm of the operator from L'^{rt) L^(fi) to itself. 

Remark 6. We have the same result if we assume Imn < and Imn ^ or if n{x) = ni{x) + 
in2{x)/k, n2{x) < and n2 ^ 0. 

In the context of non self-adjoint operator, the spectrum is not the most relevant notion. Actually 
Davies [11] introduced the notion of the pseudospectrum. Roughly speaking this set is defined by 
the points z where the resolvent is large. This notion is related with the notion of ill-conditioned for 
the matrix. Here we use the result proved by Dencker, Sjostrand and Zworski [12] to obtain a lower 
bound on the norm of resolvent. 

Theorem 6. Assume there exist ^0 G and xq €z such that lm{{n{xQ)){^o., dxn{xo))) 7^ 0. Then 
for all N > 0, sup{]fc]~^]|_Rfc2]|, fc^ ~ r{n{xo))~^ ,r > 0} = +00. Moreover if n is an analytic function 
in a neighborhood ofxQ there exists C > such that sup{e^'-^l'"'l ]|7?fc2 ]| , fc^ — r{n{xo))~^ , r > 0} = +00. 

Remark 7. Even if the lower bound, in analytic case, is of the same type of the upper bound 
obtained in Theorem 5, we cannot apply both theorems for the same direction z. Actually k^ is in 
general in C, if we want apply the Theorem 5 we need n(.To) G M, that is Imn(a;o) = 0, as Imn(a;) > 
then we cannot have lmdxn{xo) 7^ to apply the Theorem 6. 

1.2 Ouline 

In the Section 2 we prove the main technical results. Roughly speaking if the data (/, g) are more 
regular in TT^ norm we prove that the solution (u, v) is also more regular. More precisely, we prove 

for p>0, that if {f,g) e H'^'"^'^{n) ®H^^{n) then R,{f,g) = {u,v) £ H^''^'^{n) ®H'^''^'^{n). This 

2 

proves first that is compact as an operator from H (fl) © L^(il) to itself and i?f is an operator 

from Tl'^in) ® L'^{n) to Tl^^^^ {^) © H^^{n). This imphes that Rp is an Hilbert- Schmidt operator 

on H (n) © i^(r2) if 4p > n. To prove the regularity result, first we prove an estimate on u in the 

subsection 2.1. It is an easy estimate to obtain as u^g^ = 0, u satisfies a classical Dirichlet problem. 

Second we prove in Subsection 2.2 the regularity of v in all compact in fJ. As v satisfies an elliptic 

equation, far away the boundary of O it is a classical result. In the third Subsection 2.3 we prove the 

regularity result on v in a neighborhood of dfl. The idea to do this is to explain v by the unknown 

traces of v. This description allows to obtain a relation between v^qq and dnV^ga. Then we can 

use this formula on v in the equation on u. The fact that u^gn = and dnU^gn ~ gives another 

relation between v^g^ and d„v\gQ. These relations allow to determine v^g^ and with {f,g). 

This explicit formula, in sense of pseudo-differential calculus, allows to prove the regularity result. 

We need also following the same way to prove an estimate of the norm of v by the norm of /. 

This implies a weak convergence result. Actually the problem is that v has the same regularity than 

/. In particular if we consider the resolvent as a operator from L^(57) © i^(f2) to itself, we cannot 

2 

prove that the resolvent is compact. Here we avoid this problem by the assumption that f £ H (fi). 

In the Section 4 we recall some result proved in Agmon [1] and we apply this to prove the 
Theorems 2, 3 and 4. 

In the Section 5 we prove some a priori bound on the resolvent. In Subsection 5.1 we prove an 
upper bound on the resolvent near the real axe when the imaginary part of the refraction index have 
a sign and is not identically null. The main tool is to use the interpolation estimate obtained from 
the Carleman estimate. In the Subsection 5.2 we use the result obtained by Dencker, Sjostrand and 
Zworski ]12] on the pseudospectra to obtain lower bound on the resolvent. Roughly speaking this 
result says that when the operator is not elliptic in the semi-classical sense, even if a point is not in 
the spectrum, the resolvent is generically large. 

As we use deeply the semi-classical pseudo-differential calculus in the Section 2, in the Appendix A 
we fix the notation used in the rest of paper, we recall the classical results used, we give some ideas of 
proof on the action of the pseudo-differential operators on H {fl) spaces, we give some computations 
on integrals to obtain some explicit formulae used in Subsection 2.3. This allows to give the explicit 
first term of the resolvent in sense of the semi-classical pseudo-differential calculus. 
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2 Regularity results 



We describe now the idea of the proof. As we want prove an estimate when \z\ is large we will 
compute the resolvent in semi-classical framework. We multiply the Equations (4) by /i^, we denote 
hy ^ = —h?z where ji belongs to a bounded domain of C, a = 1/(1 + m) and V = m/ (1 + m). We 
change (/, g) in (-/, -g). _ 

Remind the assumption on m, we have m{x) ^ —1 for all .t G and rn[x) ^ for x in a 
neighborhood of dVL. 

Thus following (4), we obtain the system 

r { - aK^ A - ^i)u - K^Vv = f in ^ 

I {-h^A - fi)v = h^g in n (6) 
[ u = d^u = on dri. 

The goal of this section is to prove the following estimates if s > 0. The result is given using the 
semi-classical iJ* norm, see Appendix A for the definition of these spaces. 

Theorem 7. We assume that for all x G Vl, and all ^ G K", a(x)|^p — ^ and \^\^ — fi =/= 0. 

2-|-s s 1-f-s 

Let s > 0, there exists Hq > such that for f G H (fi), g G Hg^{n), u G H^^ (fl) f] Hq{Q,) and 
V € Hg^{i^) solutions of system (6) then u G H^^ (Q) , v € H^^ (W) and for h G (0,/io) we have, 



ll"ll77j+=(0) < ll/ll77;^^Q) + f^^MHim- (8) 

First we prove an estimate on u. For this, we work globally in Q. The estimate on v is more 
difficult to obtain. In a first step we prove an estimate in the interior by usual pseudo-differential 
tools. In a second step we prove the estimate in a neighborhood of the boundary dfl and we finish 
the proof. This will be do in the following three sections. 

In the proof we use semi-classical pseudo-differential calculus. We give in Appendix A, the results 
used, trace formula, action of pseudo-differential operators on Sobolev space, the parametrices. 

2.1 Estimate on u 

The goal of this section is to proof a weak version of (7). 

Lemma 2.1. We assume that for all x E fl, and all ^ G M", a(x)|^p — // 7^ 0. There exists ho > 
such that for s > 0, for all f G i/^j,(r2), v G H^^{Q,), and u G i/g (f2) solution of 

( - ah^A - y)u = K^Vv + h^f in fl 
u ~ d^u ^ Q on dQ, 

then u G i/gc (^) ^'^'^ ^ £ (O7 ho), 

ll"lk+=(a)^^'ll/ll7?L(n)+/»'ll«ll77L(0)- (9) 

Proof. As u G ffp (17), we can extend m by in the exterior of and u satisfied the same equation 
in the whole space. Here we extend also v and / by 0, this makes sense at least in L^. We have 

( - ah^A - ^i)u = h^Vv + h^l in W\ (10) 

where we denote, for w G L'^{ft), by 



w{x) 



w{x) if X G 
if X ^ 
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Let N > s + 2, we take a parametrix Q of —ah^A — /i, this is possible because we assume for 
all X en, a(a;)|^p - A* 7^ 0. We have Q{-ah'^A - ^i) ^ x + hK where x € '^^o°°(R") , x = 1 in a 
neighborhood of fi, is of order —N and Q is of order —2. 

Applying Q to Equation (10) we obtain, 

u + /i/v M = /i^Q/ + /i^Q (Fu) 
As Q is a mapping on the Sobolev spaces (see (56)) we obtain, 

We can absorb the term ft,||uj|^2(s^) by the left hand side and this imply (9). ■ 

2.2 Estimate on v in interior of Q 

To prove an estimate on v in interior of we follow essentially the same way than in the proof of the 
estimate on u given in the previous section except that we cannot extend v in exterior of U, but we 
use semi-classical pseudo-differential calculus in open relatively compact in fi. The estimate proved 
is given in the following lemma. 

Lemma 2.2. We assume for all ^ e M", we have — /.( 7^ 0. Let x £ '^^q°°(R") supported in W 

relatively compact in Q,, and s > 0, there exists /iq > such that for g G Hg^{{l) and v € H^^ (51) 
solution of 

{—h^A - fi)v — h'^g in il, 
then V £ H (M^) and for h G (0, /lo) we have, 

\\xv\\Hi+' - Hv\\H'+\n) + ^^h\\Hl^(n)- (11) 

Proof. In the sequel we can take x supported in or x = 1 on 14^ and supported in a compact 
of fl. We can essentially repeat the proof of Lemma 2.1 given to estimate u. 

As we have assumed that — ^ 0, we can take a parametrix Q of {—h^A — fi) defined globally 
in M", such that we have Q{—h^A — /i) = X + ^^'^ where K is of order —1, Q is of order —2. Let 
Xj S J = 1, 2 supported in a compact of 51 where Xi — 1 on the support of x and X2 = 1 

on support of xi- By pseudo-differential calculus we have Qxi{—h^A — /i) = X + hK-i where K-i 
is of order —1. Now we have Qxi{—h^A — /i)x2 = X + hK-iX2- We can localized the equation on v 
and we have xii^h'^A — /i)x2W = h^XiQ- Applying Q to this equation we obtain 

Xv + hK.^{X2v)^h^Q{Xia)- (12) 
Taking the H'^^^ norm of x'w we obtain (11). ■ 

2.3 Estimate on v in a neighborhood of the boundary 
Proof of Theorem 7. 

Taking account the Lemma 2.1 and 2.2, to acheive the proof, we need an estimate on v near the 
boundary 911. It is well-known that we can find in a neighborhood W of the boundary 911 a system 
of coordinates such that the Laplacian can be written 9^ -I- R{x,dx') + a{x)dx„, where x' are the 
coordinates on the manifold 951, x„ S (0, e), ^'(14^) = 9S1 x (0, e), ^' is the change of coordinates and 
i? is a differential operator on 9S1 of order 2 depending of the parameter x„ . 

We keep the notation u, v, a in the coordinates x instead oi uo'^ , etc. The Equations (6) become 

{aiDl^ + R{x,D') + haDxJ - n)u - K^Vv = h^f in dD x (0,£) 
{Dl^ + R{x, D') + haDx„ - t')v = h^g in dD x (0, e) (13) 
u = djyU = on dfl x {0}. 
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We have taken the semi-classical notations, D^^ = jd^j , D' = [D^^ , • ' ' 7 Actually R and V 

may depend explicitly of h but this introduces no problem in the estimates, if V\h=Q 7^ 0, this is the 
case if n = ni + inijk^ we have V\h=^^(x) = (1 — ni)/ni 7^ by assumption. 
Let w £ L'^{xn > 0), we denote by 

/ w if .T„ > , . 

2« = l.„>ou;=| oif^^<o, (14) 

Usually we use w but some time it is more convenient to use lx„>ow. We have 

Dx^w = ^x„>oDx„w + -w\x„=o ® Sx„=o 

Dl W = lx„>oDl W + ® Sx„=0 + -W|:r„=0 ® D^^5x„=0- 

Here and in the sequel, for simplicity we denote by w\x„=o the limit, when x„ goes to with Xn > 0, 
of w{x' , Xn), if the limit exists. Here the distributions u and v are solutions of elliptic equations then 
the limits exist in a space of distributions. 
From (13), we obtain 

{a{Dl^ + R{x,D') + haDxJ - ^j)u- h^Vv = h^l'm dil x {~e,e) 

{Dl^ + R{x, D') + haDx^ - = h'^g + t7o ® Sx„=o + jji «) £'x„4„=o in dn x (-e, e), 

where 70 = Dx,^v\r^^^Q + ahv\x„=o ^-nd 71 = U|x„=o- We can consider these equations for a;„ G (— e,£) 
indeed the coefficients of R are smooth up the boundary, and we can extend R in a neighborhood of 
the boundary for a;„ < 0. The functions u and v are null for a;„ < so the equations are relevant 
only to take account the boundary terms. Remark, in the first equation because the traces of u are 
null, they are not boundary terms. 

The main goal of this section, is to obtain estimates on 70 and 71 . 

Now we search, using the equations (15), two relations between the traces of v. First we localize 
V in a neighborhood of the boundary. We denote by w = xo^ where xo S ^°°(K), Xoi^n) = 1 in a 
neighborhood of boundary, for instance if \xn\ < e/4 and Xoi^n) = if |a:„| > e/2. From the second 
equation of (15) we obtain 

{DI + R{x, D') + haDx^ - fi)w = h^xog + -70 ® 5x„=q + -71 ® DxJx„=o + hKv on dn x K, (16) 

— I I 

where K is a first order differential operator coming from the commutator between Z?^^ or Dx^ and 
Xo- 

Let xi G ^°°(K) such that XiXo = Xo for instance Xii^n) = 1 if \xn\ < e/2 and Xo(a;n) = if 
\xn\ > 3£/4. By assumption we have + R{x,^') — /i 7^ then by semi-classical pseudo-differential 
calculus there exists Q of order -2 such that Q{D^^ + R{x, D') + haDx^ ^ A*) = Xi + hK where K 
is of order —N where N > s + 2. Applying Q to (16), we obtain 

~ h h 
Xiw = w = Q(-7o ® Ox„=o + -71 ® Dx„Sx„=o) + gi 

where gi = —hKw. + h^Qixos) + hQKv thus 

\\9i\\Hi^\n) - f^MlIlt'in) + (17) 

actually we can estimate Kw because w G L^(IR") and K is smoothing. By this trick we have not 
to verify that K is a mapping on the H^^. In appendix A, Estimate (56), we have proved that a 
parametrix as Q is a mapping on the H^^{n). 

By ellipticity assumption on -I- R{x,S.') — there exist pi{x,£,') and ^2(2;,^') with Impi > 
and Imp2 < such that ^l + R{x,C) - M = (?n - Pii^ADi^n - P2ix,C)) (see A.2). 

From (59) and Lemma A.l we have 
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Q(^7®i^l4„=o)-op(g)7 (18) 

I giXn^n / h ^^n 



where g - — / e""^"/"- -d^n + r-2+k, (19) 

where we denote here and in the sequel by rj an operator of order j . 

From Lemma A. 2 we obtain if we restrict (17) on {x„ = 0} and as w^^^^o ^ 71: 

71 = op( )po + op( — — — )7i + /iop(r_2)7o + hop{r^i)ji + (.gi)u„=o- (20) 

Pi - P2 Pi - P2 

Thus we obtain 

op ( — — — ) 71 +op ( ) 70 = {gi)\x^=o + /iop(r_2)7o + ft.op(r_i)7i. (21) 

\P2~P1J \P2-P1J 

Then applying op(p2 Pi) on both sides of (21), using estimate (17) to estimate gi, trace formula 
and by pseudo-differential calculus we obtain 

op(/02)7i + 70 = 92 where 32 = op{p2 - pi){gi)\x„=o + hop{r^i)jo + hop{ro)ji 

and |.92|^|+i/2(gf^) < + h'^^^hW Hl^in) + '^IioIhi-^z-^i^ou) + ^ItiI^i+i/^oo) (22) 

To obtain a second equation on the traces, we use the first equation of (15). As before there exist 
Q of order -2, K of order — iV— 4. X2 such that X2X0 = Xo and such that Q{a{D^ + R+ haD^^) — p,) — 
X2 + hK . 

We apply Q to the first equation of (15), we obtain 
X2U^h^Q{Vw)+g3 

where 53 = -hKu + h^Q{V{l - xo)v) + h^Qf 

ll53ll7r;+*(o) ^ ^Il"ll7r:+^(n) + /I'lKl - Xo)w|l7j=+2(f^) + h^\\f\\jj^+2^^y (23) 

Using Lemma 2.2 we can estimate the term {l—Xo)ll; using the Lemma 2.1 we can estimate ||w||;y»+2j.j^^ 
and we obtain. 

We replace w by the Formula (17), we take the trace on x„ = 0, as u\x„=o — Oj we obtain 



= 54, (24) 

|x„=0 

where ,94 = -/i"^(.g3)|i.„=o - [QV{gi)]i^^=o, then 

As a{x){F,'^+R{x,C)) — P is elliptic, the polynomial in ^„ has two roots Xj one satisfies ImAi > and 
the other ImA2 < (see section A. 2). Wehave a{x){£,f^+R(x, £,'))— p = a.{£_n — \i){(^n — X2). The prin- 

VX1X2 



cipal symbol of Q is a(i„-x^m„^x,) - The principal symbol of QVQ is a(i„-x,m^-x,m„~p,m„-p,) ■ 
Following the same method used to obtain (21) from (17), we have by (59), Lemmas A.l and A. 3, 
as X2 = 1 in neighborhood of dfl 

f V^(A2-Ai+p2-pi) \ 

\a{\i - A2)(Ai - P2)(pi - A2)(pi - P2)) 

f ^(^2^2 - PlAi) A _ 

\a{\i - A2)(Ai - p2){pi - A2)(pi - P2)) 
where 55 = 54 + /iop(r_4)7o + /iop(r_3)7i. 



9 



As 7^ in a neighborhood of d^, we can apply op(pr(Ai — A2)(Ai — P2){pi ~ ^2){pi ~ P2)), we 
obtain 

op((A2 - \i+ p2- Pi)) 70 + op((p2A2 - piAi))7i = ,96, 

where ge = op(-^(Ai - A2)(Ai - p2)(pi - A2)(pi - P2))ff5 + ^op(ro)7o + ft.op(ri)7i 
then |56lff|-i/2(gfj) < /i'^'ll"ll7i-^+i(n) +^"'^'11/1177:+^^) 

+ ^^1701^1-1/^(9^) + ^l7ilff|+i/^(aa) (26) 

Now we have two equations on traces by (22) we can replace 70 in (26). We obtain by pseudo- 
differential calculus 

op(p2A2 - Aipi)7i - op((A2 - Ai + p2 - Pi)p2hi = .96 - op(A2 - Ai + P2 - Pl)52 = 57- 
This implies, 

op((p2 - Pi)(Ai - P2))7i = 57, with 

+ /l|70|^|-l/2(gf5) + hhl\jjs + l/2^g^y (27) 

As ImAi > 0, Im pi > and Imp2 < 0, the symbol (p2 ~ Pi)(Ai — P2) is elliptic, by inversion we 
obtain 

l7ll^|^3/.(,^) < +/,3/2|,^|j_^^^^^^ ^;^-l/2,|^||_^^^^^^ 

+ '^l'Tolff|-i/=(an) + '^hi\H;r^\onr (28) 

and using (22), we obtain 

l7ol^j+v.(9^) < /^'/'lkll7r:+^(o) +/^"'/'ll/ll7r:-(n) 

(29) 

Summing (28) and (29) we have for ho small enough 

From (17) and from estimate (57) obtained in Appendix A 

MlTJi+^t^n) < 11.91 Il7?:r + '^'^' (lTolH|,+i/^(ao) + l7ilH|.+3/^(on)) 

We can now estimate v, we have by (30), Lemma 2.2, 

MH:t\n) ^ Il^ll7?:r(n) + ll(i-xo>ll7?:r(o) 

This implies 

MH:t'in)^'^^y\\H:jn) + \\f\\Hit\nr 

if ho is small enough. Using this estimate and (9) we obtain 

These two last estimates imply the Theorem 7. ■ 
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Remark 8. Actually in this proof we have assumed v € H^^ ($7). To prove the result with v G 

' s+l 

_ffg^(ri), we argue in two steps, first following the same proof we can obtain v G H {Q) and second 
the proof given above gives v G H'l^^{il) 

The estimate proved above are not enough on v with respect /. For the sequel we need to estimate 
V hy f in H'^ norm. 

Proposition 2.3. We assume that for all x G fl, and all ^ G W\ a(x)|^p — /i 7^ and — /i ^ 0. 

There exist Hq > and C > and for all S > 0, let xs G supported in a S -neighborhood of dQ, 
there exists Cs > such that for h G (0, ho), f G Tll^iil.), g = 0, uG n H^{^), v G Illci^) 

and Av G L'^{n), solutions of system (4) we have the estimate, 

MiTUn) < C\\xsf\\Hi^^n) + Csh\\f\\Tjs^^^y 

Proof. The proof follows the previous one. We give only the modifications to do. From (16) we 
obtain (17) with the estimate 

Thus we obtain (21) and (22) with the estimate 

|52|H=-3/2(gf2) < I Op(ri)(gi)|gf2|ff=>-i/2(an) + ^l7o|H=-5/2(ao) + 'i|7l|H=-3/2(gn) 

< /i^/^||u||77=jn) + /»l7o|if-5/2(ao) + /i|7i|h— V2(9o)- (32) 

We must modify (23) to obtain the term xs- We take the same Q as in (23) and we apply Qxs to 
the first equation from (15). We obtain 

Qxs {a{Dl^ + R{x, D') + haD,^ ) ^ l^)u- h^QxsVv = h^Qxsf_ in dn x (-e, e). 

We have xs{a{Dl^ + R{x, D') + haD^J ^ m) = («(-D^„ + Ri^i D') + haD^J - fi)xs + Li where Li 
is a differential operator of order 1 depending of 6. As Q{a{D^^ + R + haD^^) ^ /^) = X2 + hK^M 
where K-iq is of order —N, with > s + 2, and X2XS = Xs-, we have 

Xs-iL = h^Q{xsVw) + 53 

where = —hQLiu + hK^]yu + h^Qixsf) 

||53||-H-^+2(fj) < C5h\\u\\jjs+i^^^^ + Ch^WxsfWni^i^n)- 

We have used that Q and Li, a differential operator, act on the hI^. We can estimate u by (9), this 
gives 

\\93\\w+-^n) ^ Csf^^'W-fWHUn) + C5/i'IK'll7?L(o) + Ch^WxsfWTJijn) 
We replace w by its value given by the formula (17) with the estimate (31). We obtain (24) with 

\9^\H!^^^\on) ^ Csh'/^fWj^s^^,,) + Csh'^'Mj^.^^,,^ + Ch-'/^xsf\\Hi^^nr (33) 

If we compare this estimate with (25) we see that the bad power of h in front of / is only a part of 
/ localized in a neighborhood of the boundary. 

We can follow the proof and we obtain (26) where the estimate on g^ is 

+ C'^l7o|_H-=-V2(ao) + C'^l7i|ff=--V2(ao)- 
We have the Formula (27) where gj, from (32) is estimated by 

+ C'^l7o|_f/=-s/2(ao) + C'/i|7i|jf=-3/2(an)- 
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By ellipticity and Formula (27) we obtain 

+ C'^l7o|ff=-5/2(af2) + C'^l7ilH=-3/2(an): 
and by (22) where 32 satisties (32). we have, 

l7o|ff.-3/.(ao) < C^^/»'/'ll/ll77:^(o) +C/^"'/'llx.7ll7?L(a) 
+ C''i|7o|ff=-5/2(af2) + C'^l7ilif=-3/2(an)- 
Summing the previous estimates and for h small enough, we obtain, 

From (17) with gi satisfying (31), we have by (57) 

IkllTT^jn) < t^li5ill7?:,(o) + Ch^^^ (l7o|^|-3/2(3j,j + l7i|^|-i/2(gf,)) 

Using the formula (12) in the proof of Lemma 2.2 with g = 0, we obtain ||xw||;fys ^jj-j < Cjlwll-g^s 
This estimate and (34) give 

lif ll7?;^(o) ^ C'<5^IK'll7?^^(n) + CsHfllL^n) + CUxsIWhi^^q)- 
This estimate is also true for a fixed 6 then we have for h e (0, ho), ho small enough 

This with the previous estimate implies the Proposition 2.3. ■ 

3 Existence and compactness 

In this section we prove the Theorems 1 and 2. 
3.1 Proof of Theorem 1 

Proof. We follow the proof given by Sylvester [23, Proposition 10], we prove that le range of is 
closed and dense. 

To prove the range is closed we apply the a priori estimates prove in section 2. We recall that 
a = 1/(1 + m) and V = m/{l + m). 

We remark that if we have Ce C then Ce U (—00, 0] ^ C. Indeed, as H is compact, Cg is closed. 
If Ce U (—00, 0] = C then Ce \ (—00, 0] = C \ (—00, 0] as C \ (—00, 0] is dense in C, we have Ce = C. 
Let zo such that zq ^ Ce U (—00, 0], we can choose \zo\ = 1, let z — h^'^ZQ we have /i = — zq- First we 
can estimate ||w||l2(q) by Proposition 2.3 with s = and S fixed if g = and by Theorem 7 if / = 0. 
There exists C > such that for all \z\ large enough, 

\\v\\mn)<C\\f\\L2^n) + i^\\g\\LHn)- (35) 

We can apply the Lemma 2.1 with s = 0, we obtain with the previous estimate on v, 

C 

\A''MLHn) + + 11^11^2(12) ^ + j^Mmn)- (36) 

Clearly these estimates prove that the range of Bz is closed where the norm on the domain of the 
operator is given by the norm for u and by ||w|| + ||Au|| for v. 
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To prove the density of the range of we prove that the orthogonal of the range is {0}. We 
recall the Green formula, if v and q are smooth functions in f2 we have 

{v\Aq) - {Av\q) = {v\d,q)o - {d,v\q)o, (37) 

where (-j-) is the inner product in f2, (■|-)o is the inner product on dfl and 9^ is the exterior normal 
derivative on dfl. Actually (37) is true if v smooth, q G L^(fi) and Aq E L'^{il). Indeed, in this case 
it is well known that q\QQ £ H~^^'^{dfl) and d^q^gn g H^^^^{dfl) then we can find /„ and f/„ smooth 
functions such that /„ goes to Aq in L'^{il) and <?„ goes to q\dn- Let qn the solution of Ag„ = /„ in SI 
and {qn)\dQ = 9n, qn is a smooth function and by continuity {duqn)\dn goes to di,qon in H~^^'^{dQ). 
Then we can pass to the limit in (37). 

Let p,q G L^(f2) and u, v smooth functions in fl, if (p, q) is in the orthogonal of the range we have 

{Au - z{l + m)u + mv\p) + {Av - zv\q) = 0. (38) 

We take u,v £ '^Q°°(ri) in (38), by integrating by part in distribution sense we have 

Ap-z{l+m)p^Qinn (39) 
Aq - zq + mp ^ in fi. (40) 

In particular Ap and Aq are in L'^{fl), then we can apply (37) to integrate by part in (38) if now 
u, V are smooth functions up the boundary with u^gn — di^u^g^ ~ 0. Using (39) and (40) we have 

{v\di,q)o - {d^v\q)o = 0. 

As v^QQ and O^v^qq are arbitrary, we obtain q^QQ = duq\Qn ~ 0. By (40), q satisfies a Dirichlet 
boundary value problem and d,yq\QQ = 0, then q G HQ{rt). By (39), Ap G L'^(ft) and p € L^(f2). We 
deduce that {q,p) G ^^o(^) ® {v G L^(r2), At; G L^{il} satisfies the same kind of equation as {u,v). 
Then the inequalities (35) and (36) prove that p = q — 0. This acheives the proof of Theorem 1. ■ 



3.2 Proof of Theorem 2 

Proof. We take the same z as in the proof of Theorem 1. We can apply Theorem 7 with s = , we 
obtain in classical norm 

11 C 



CI c 
\v\\mn) + TnMH\n) + g2ll^ll-f?^(o) ^ + ^II.9IIl^(o)- (41) 



This proves that : H^{n) © L^{n) H'^{n) ® H^{n), then is compact from H^{n) ® L'^{fl) 



to itself. We can apply the Riesz theory. 



4 Spectral results 

Here we prove how the regularity results obtained in the section 2 allow to prove the spectral results. 
Actually the result obtained in Theorem 2 is not enough to prove that the spectrum is a countable set. 
The theory given in Agmon [1] is based on the spectral decomposition of Hilbert-Schmidt operators. 
We adapt two results given in Agmon to our case, the Lemma 4.1 and the Proposition 4.2. Following 
these results we will prove the Theorems 3 and 4. 

Let T an Hilbert-Schmidt operator from H {fl)®L'^{i}) to itself. We denote by |||r||| the Hilbert- 
Schmidt norm. Let ((^j)jgN a Hilbert basis on H (fi) and {ipk)keN a Hilbert basis on L^(f2), then 
(((?!)j,0))jgN, ((O,-0fe))fc6N is a Hilbert basis on F^(ri) © L'^{Q). Let r(0j,O) = uj = (w°,m]) and 
T(0,7Afe) ^Vk = (vlvl). With these notations, we have |||r|||2 = E.'lodl^jlllr^fo) + ll^jllir^(o) + 

h]\\hm + \\v]\\hm)- 

We denote by \\T\\j the operator norm from H^{n)(BL^{n) H^^^{n)®H\n), where H°{n) = 
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2 

Lemma 4.1. Let m > n/2, there exists C > such that if T is a bounded operator from H (17) 
L2(17) ^ H ^ {n)®H {n), then T is a Hilbert- Schmidt operator and 

\\\T\\\ < c||r||;^/(2'")||T|jJ""/('"\ 

Proof. We follow the proof given by Agmon [1, Theorem 13.5]. 

Let u = T(Ef=o«. (^. .0)) +r(Ef=o^.. (0,^.,)) = Ef=o + Ef=o = We have 

Uj = {u^,u}j) and Vj ~ We treat the term u". 

If m > n/2, 77™ (r2) C L°°{n) and (see [1, Lemma 13.2]), for a e N", |a| < 2, there exists C > 
such that 

V(2m) 11^,11 l-n/(2m) 

By the property on T, we have 



< C\\T\\lY.{\a,\' + |6,f ) and < ^(|a,f + |6,f ). 

Let i^: = ||T||"/'^"'||T||o""/^^'"^ thus we have for x G fl, and for a G N", |a| < 2, |9"w0(x)|2 < 
CK^ T,f=oi\aj\^ + l^jf )• We have d'^u^ix) = T,f=oi'^39°'u'^j{x) + bjd°'v°{x)), we take in the previous 
inequality aj = d°'u^{x) and bj — d"rP^{x), we sum on a, we obtain for all x £0,, 

(N \ ^ N 

^(|9"«^"(x)p + |a"t.,"(.T)n < ^ 5](|9"u°(x)p + Wv^,{x)\^). 
i=0 / |q|<2j=0 

Thus Y.\a\<2{\9°'^%^)? + |9"w°(x)p)) < Ci^^ integrating this on O (which is bounded) we find 
E^Lndl'^ill— 2 + Ik? II— 2 ^) — CK^. As the right hand side does not depend on N we can let 

goes to infinity. We can treat by the same method the terms E^o(II'"]IIl2(si) + 11^^)111.2(^2)), it 
suffices to repeat the previous proof without the derivative terms. This means that |||T||| is bounded 
by CK = C||r||;^/(""'||T|!j-"/(""'. ■ 

We give here a small improvement of the Theorem 16.4 in [1]. 

2 

We introduce some notations. The inner product in H (£7)0X^(17) will be denoted by (-j-). Let T 
an operator from H (ri)0L^(il) to itself, if A"-^ is in the resolvent set of T, we set T\ = T{I — \T)~^. 
We remark that if T is the resolvent of P, that is PT = I, then T\ is the resolvent of P — XI. Indeed, 

{P - XI)Tx = {P- XI)T{I - XT)-^ = {I - XT)-'^ - XT{I - XT)-^ = {I - XT)-\l - XT) = I. 

2 

Proposition 4.2. LetT a Hilbert- Schmidt operator on H (Q,) (B L'^ {^) . We assume that there exists 
< 01 < 02 < ■■ ■ < On <2t: such that 0k - 0k-i < tt/2 for k = 2, ■ ■ ■ ,N and 2tt-0n + 0i < 7r/2 
satisfying there exist rg > 0, C > such that sup^>,,^^ \\T^^ie^. ||o < C, for k ^ 1, ■ ■ ■ , N . Moreover we 

assume there exists (Xj) such that \Xj\ — > +oo and for all f and g in H (Q,) L'^lil), {Tx.f\g) 0. 
Then the space spanned by the non zero generalized eigenf unctions of T is dense in the adherence of 
the range of T. 

Proof. As in Agmon ]1, page 284] we define F{X) = {Txf\g) where g is orthogonal to the 
generalized eigenf unctions. The goal is to prove that F{0) = 0. As in Agmon we can prove that F{X) 
is analytic in C and bounded. Then F{X) is constant by Liouville theorem and as {T\ .f\g) — > this 
implies that F{X) =0. ■ 

Proof of Theorem 3. 

Before the proof we give some results on the links between the spectral decomposition of S and 

2 

S^, where S is an bounded operator on H (17) L^(i7). 
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Let uij for j = 1, • • • ,p, the roots oi = 1. We have zP — 1 = Y[^=ii^ ~ ^j)' particular for 
z = we have — 1 = n?=i(~'^j)- Thus we have 



p p 



l^Uiz u,) = n(-^.-) 11(1 - u-'z) = Y[il (42) 

as we have {ujj, j = !,■■■ ,p} ^ {uj-^, j 1, • • • 
Applying (42) to zS, we obtain 

(l-.^^5P) = n(l-w,z5). (43) 

If (I — LUjzS) is invertible for all j, this implies that [I — z^S^) is invertible. If for a fixed j, (I — WjzS) 
is not invertible, either ker(/ — uijzS) ^ {0} this implies kcr(/ — z^S^) ^ {0} or the range is not 

H^{n) e L^{n) this Imphes that the range of (/ - zPSp) is not II^{n) ® L'^{n). We deduce that 

/ — ujjzS is invertible for all j ^ I ~ zPSP is invertible. 

If SP is compact and / — zPSp is not invertible then by the Riesz theorem z~p is an eigenvalue of 
SP and there exists k such that kcr(/ - zPSPf-^ ^ ker(/ - zPSPf = ker(/ - zPSPf+'^ and the 
dimension of ker(/ — zPSp)^ is finite. 

We will prove that all the eigenvalues of S have the form ixjjZ^^. Indeed 5 is a operator on 
ker(/ — zPSP)'', then S admits a spectral decomposition on ker(/ — zPSP)^ . Let u ^ and A such 
that u = \Su then zPu = [\zS)Pu thus \'^p{I - zPSPfu = (A^ - zP)''u and A^ = zP this implies 
A = WjZ. 

Now we prove that kcr(/ — LOjzS)^ ~ kcr(/ — ijjjzS)^^^ . From (43), we have 
u e kcr(/ - LOjzSf+^ C ker(/ - zPS'P)'=+i = ker(/ - zPSPf. 

We have 



(/ - zPSPf = {I-{I-{I~ zUjS)ff = \ P{I- zuJjS) + C^{I - zujjS) 



k 



fj,=2 



this implies (J — zPSp)^u = p'^(/ — zujjS)^u = 0, which is the claim. 

Obviously we can find the spectral decomposition of Sp from the one of 5. This proves that there 
exists j such that ker(/ - w^zS')'""^ ^ ker(J - ujjzS)^ = ker(/ - ujjzSY^^. 

To prove the Theorem 3, we fix z as in the proof of Theorem 1, we denote hy S ~ Rz and we 
apply Proposition 4.2 to T = 5^ = By Theorem 7 and Lemma 4.1, T : if {VL) © L'^{n) 
H^+^P{Cl) ® H^P{Q) then T is an Hilbert-Schmidt operator as p > n/4. We remark that 

Sx = iRz)x = Rz{I- XRz)-' - (R^' - A)-i = [B, - Xy' ^ (Bo - z - Xy' = R,+x. (44) 

As Ce is a closed cone, if re*^ is not in Ce for all r large enough, then z + re*^ is not in Ce for all r 
large enough (see (3) for the notation Ce)- 

Using the previous remark and the estimate (41) we have that HSreif || = ||i?2_|_,,ei8 ||o is bounded 
uniformly with respect r large enough, if 6' 7^ and re*^ ^ Ce- 

We prove the following formula 

p 

pzP-^r.p ^^WfeS-^,,. (45) 
fc=l 
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Indeed we take the inverse of (43) when the formula make sense, we have 



ii-zpsn-'^ll(^~''^'^y'- (46) 

Derivate (42) with respect z, we obtain 
We apply this formula to zS, we obtain 

p p 

p^P-igP-^ ^Y^ujk n (l-^j^^)- (47) 

We multiply term by term (47) and (46), multiplying the result by 5, we obtain (45). 

If S^i,z is bounded uniformly for r large enough and for all k, by (45) we have ||T2p||o < If 
we assume that Ce is contained in a sector less than 6 with 6 < 11/2 and 9 < 27r/p, the union of Ce 
and Ce rotated by angle 2fc7r/p does not give C and the Formula (45) proves that we can find the 9j^s 
satisfying the assumption of Proposition 4.2. If p > 2 the estimate on HT^pHo is stronger than the 
weak convergence. In case p = 1, we have by the Theorem 7 and Proposition 2.3 with the notation 

./=(/l,./2), 

\\RJ\\H^in)s^LHn)<(^\\xsh\\LHn)+Cs\zn\f^^^^^ 

if \z\ is large enough and z ^ Cg- For /i and e > fixed we can choose S > such that 
C'||x5/i||L2(f2) < £■ Then it is easy to prove that if Zj is on a line such that Zj — rje^^ with 
rj +00 , we have limsup ||i?^^/||-g.2^j^^^^2(Q) < £• This prove that ||-R2j/||-H-2(Q)^2.2(f2) ^ ^^^^ 
\\Szj-zf\\jj^i^^2)eL^(n) ^ ^■ 

Now we prove that the adherence of R,{H {n) © L^i^)) is H§{n) ® {v e L'^{Vi), Av € L^(rJ)}. 
Let (m,w) G H^{n) ©{we L^{n), Av e L^ifl)} we have B^iu.v) = {f,g) e L^i^) © L2(f]). Let 
{fn,gn) e (ri) © L^(il) such that {fn,gn) ^ (/, ff) in i^(ri) © L2(fi). We can take for instance 
fn and gn in ^Q°°(ri). We have by continuity Rz{fn,gn) ^ Rz{f,g) ~ {u,v) in i?o(^) © S 
1/^(0), Au e i^(f^)} with the norm defined by ||w||l2(s:2) + \\Av\\l2(^q) on {u e L^(rJ), Av G L^(r2)} 
and the usual norm on Hq{H). ■ 

Proof of Theorem 4. Using (44) as z is fixed, to estimate the number of eigenvalues less than 
is equivalent to estimate the number of eigenvalues A such that z + A is less than P . That is in 
the sequel we estimate the number of A less than such that A~^ is a eigenvalue of 5* = i?^. 

We have shown in the proof of Theorem 3 that ||Tjp ||o < \z\p-^ '^^ is on a line c?(ro, 9) ~ {z ^ 
C, z = re'^.r > ro} C Ce- As (1 - zPT)-^ = 1 + zPT^p, we obtain for \z\ > 1, ||(1 - zPT)-^\\o < 
l + \znT,4o<C\z\. 

We have, by WT^php < ||r!|2p!|(l - zPT)-^\\o < C\z\. We obtain from Lemma 4.1 

ll|r.p||| < c||r.p||^;/('*'''||r,p||;-"/(^'') < c\z\'-p+-/\ 

We remind [1, Theorem 12.14] if T is Hilbert-Schmidt, we have ^ < |||T||p where fij ^ 
are the eigenvalues counted with multiplicities. 

Let Xj such that Xj^ is a eigenvalue of S, we find that ^p^^^ is a eigenvalue of T^p. We obtain 



^ |A? - zP 



If |Aj| < and taking z e d{ro,9) satisfying \z\ = t^, we have |A^ — zP\ < 2t^P. Then we have 
Then we obtain that N{t) < Ct*+'\ 
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5 Estimate on the resolvent 



5.1 Upper bound 

In this section we prove the Theorem 5. We recall the well-known Green's formula. For regular 
functions u and v, we have 

/ (mAw — v/S.u)dx ~ I {uduV — vd^u)ds, 

where is the exterior normal derivative on dfl and ds is the surface measure on dQ. Here we work 
with smooth functions. As the problem is well-posed by Theorem 1 we can apply the estimate for 
non smooth functions by passing to the limit in the estimate. 



Aw + k'^n{x)w = / in f2, 
Aw + k'^v = g in Q, 
w = V on dQ, 
d,,w = d,,v on dn. 



By Green's formula and (48) we have 



{wAw — wAw)dx — / {wd^w — wduw)ds 
n Jdn 



w{f — k nw) — w{f — k nw)]dx 
[wf — ijjf — 2ik'^ lmn\w\'^]dx, 



n 



and 



(48) 



(vAv — vAv)dx = / {vdi/V — vdi,v)ds 

2 

= j [vg- vgjdx. 
Jn 

Using the boundary condition in (48), we obtain, 

M--^f)dx- ^i.g-,g)dx = 2^^k^[M^l.^ndx. 
n Jn Jn 

Thus we deduce, 

S I e[w[^ <[\v[\[\g\\ + [\w[m, (49) 

where w = {x G fi, Imn(a;) > 5}. 

Remark 9. In the case where n = ni+in2/k we have Imn ~ n2/k, and in the previous computations 
we must change the left hand side of (49) by S k[w['^ where lu = {x E fl, n2{x) > S}. We let to 
the reader to check that the rest of the proof does not change with this new estimate. Indeed the 
powers of k do not play any role with respect the estimates by e'^'^ . 

We recall the interpolation estimate. We can find this type of estimate in [20, Section 3, Formulas 
(1) and (2)], [21, Theorem 3[, [10, Proposition 1.2]. The estimate (50) does not appear in this 
literature, but we can prove it following the same ways. Indeed in the Carleman estimate used 
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to prove the interpolation estimates, we estimates also the boundary terms but in the previous 
mentioned paper we did not need the boundary term in the interpolation estimates. 

Let X = (-3, 3) X 17, y = (-2, 2) x Vt, and O = (-1, 1) x uj. We denote by dY = (-2, 2) x dVL. 
Then there exist 5 > and C > such that for ah W c^H^iX) such that d^W + At¥ G L'^{X), 
W\QY € H^{dY), d^W\QY e L'^idY) , we have 

IIW^II \d.W\0Y\mdY) <C {\\dlW^ ^W\\L^x)+\\W\W-^^o)Y\m}^,'^^^ (50) 

||W^li77i(y) <C{\\dlW + AVK|U2(;,) + iM^iayUnay) + \d,W\0Y\mdY)Y\\Wt-,'^^^, (51) 

where s is an additional variable. This variable allows us to give an estimate uniform with respect 
the large parameter k. We shall see that in the sequel. 

Let W{s, x) = e'^^w{x) where Au; + k'^w = / in f2. We have d^W + AW^ = e'^^ f and we can 
obtain the following estimates for a C > 0, 

\\M\H^(n) < C\\W\\tj.^^^, 

|9i/W|onlL2(aj2) < C\d^WiQY\L^(dY), 
\\d',W + AW\\mx)<Ce'''\\f\\LHn), 

\\w\\mo) < Ce'^Mmu:), 

By the interpolation estimate (50), there exists C > such that for all w e H^{Q), satisfying 
di^w\gQ e L^{dil) and G H^{dil) solution of Aw + k^w — f in ft, we have 

\{d„w)i9n\LHdn) + \w\an\m{dn) + W^Wj^if^^-^ < Ce^''{\\w\\L^cj) + ||/||). (52) 
Using (49), (52) and Ce'^'^|lw|| ||/|| < (l/2)||w||2 + C^e^^''\\ff, we have for a C> 

Following the same way, we denote by W{s, x) = e''''v{x) and we apply the interpolation estimate (51), 
we obtain on v the estimate 

\H\H\n} - Ce^'^dlsll + \v\dn\midn) + \duV\on\L^dn))- 
Taking account the boundary condition in (48) and (52) we have 

<ce^H\\9r+\\.fr + \Mhi.)) 

<Ce^'=(||5|P + ll/f + lK'||||.gil + ||HIII/ll)by (49). 

This estimate and (53) give 

ll^lli?^(o) + !I^HlH^(o)<CeC'fc(ll/ll + 11.911)- 

This implies the estimate on the norm on v and w, which gives the Theorem 5 by density. 



5.2 Lower bound 

Here we use the results proved first by Davies [11] in one dimension, by Zworski [24] for Schrodinger 
operators in n dimension and by Dencker, Sjostrand and Zworski [12[ for more general sub-elliptic 
operators. This allows to obtain a lower bound on the resolvent. 

We recall here the theorem given by Dencker, Sjostrand and Zworski. 
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Theorem 8 (Theorem 1.1 [12]). Let V G Then, for any z € {^'^ + V{x), (.t,^) G 

lTa.{£,^dxV{x)) 7^ 0}, there exists ho > such that for all h G (0, /iq), there exists u{h) G L^(R") wif/i 
the property 

\\{-h^/^ + - z)u{h)\\ = 

/n addition, u(h) is localized to a point (xo,^o) *^ phase with + y(xo) = z. 

More precisely, WFh{u) = {(a^OiCo)}; where WFhiu) is the semi-classical wave front set. 
If the potential is real analytic, then we can replace h°° by exp(—l/C'h). 

A consequence of the microlocal locaUzation of m. we can cut-ofF u such that its support is in a 
neighborhood of xq. The Theorem 8 imphes that if z is in the resolvent set, \\{~h'^A + V{x) — z)^^\\ > 
CNh~^ for all TV, in <r°° case and \\i-h^A + V{x) ~ z)-^\\ > Cexp{C/h) in analytic case. 

Proof of Theorem 6. We set zq ~ — (1 + m(a;o))^^ |^oP; we set V{x) = zo{l + m{x)). We have 
|?o|' + ^^(2:o)-0and 

lm{£,odxV{xo)) = lm{zo£,odxm{xo)) = -\1 + m{xo)\~'^\£,of lm{{l +m{xo))£,odxm{xo)) 
= -|l + m(2;o)r'|Co|'lni(n(a;oKo5."(xo)) ^0, 

by assumption. By Theorem 8 there exists u{h) such that \\{—h'^A + V{x))u{h)\\ = 0{h^)\\u{h)\\ 
or = ©(e"'-^/''-' if m is analytic. Define by / = Au{h) — h^^zo{l + m{x))u{h), we have 
i?/j2(/, 0) = {u{h),0) with fc^ = —h^'^ZQ. We remark that is localized in a neighborhood of xq, 
in particular is null in a neighborhood of dfl, then (u(ft,),0) satisfies the boundary conditions. 
This implies the Theorem 6. ■ 



A Notation and recall on pseudo-differential calculus 

A.l Sobolev spaces and pseudo-differential operators 

We introduce some notation for the Sobolev spaces. 

We denote the semi-classical i?" norm by = /(I + h'^\^\'^y\'u{£,)\'^d£,. On a compact 

manifold we define the semi-classical ff" using local coordinates. To distinguish norm on spaces of 
dimension n and dimension n — 1, we denote the semi-classical iJ'* norm on dil by \-\h= (dn)- Let w a 
distribution on ft, we denote by Ijuilj-^s ^^^j — inf{||/3||jys^, where /3|f2 = w}. We recall that we denote 
hy D = {h/i)d, and if s is an integer the quantity X]|a|<i5 11^"^' is equivalent uniformly with 

respect ft. to 

In the context of semi-classical H'^ space we have the following trace formula, for s > 0, for all 
w G H^^^^'^{^) we have 

where ^[^^^(^^o) means the limit of w{x) when a; G f2 goes to xq. 

We recall the pseudo-differential tools. Let a{xX) in "^""(R" x M") we say that a is a symbol of 
order m if for all a, /3 G N", there exist Ca^p > 0, such that 

|9,"9fa(x,e)|<C„,^(0"-l^l 

where (^)^ = 1 + In particular a polynomial in ^ of order m with coefficients in "^""(M") with 
all bounded derivatives, is a symbol of order m. 

To a symbol we can associate an semi-classical operator by the following formula 

Op{a)u^a{x,D)u^j^ J e^-^ a{x M)u{m = J e^^^"^a{x,Ou{i/h)d£,. 

This formula makes sense for u G ^(M") and we can extend to u G iJ*. For a, a symbol of order m, 
there exists C > such that for all u G , 

\\a{x,D)u\\jfs-^ < C\\u\\h;^. 
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We can compose the pseudo-differential operators, and we have the following result. Let a a 
symbol of order m and h a symbol of order k, there exists c a symbol of order m + fc such that 
a(x, D) o h{x, D) = c{x, D). Moreover there exists a symbol d of order m + fc — 1 such that c{x, D) = 
{ab){x,D) + hd{x,D). This means that up h the composition of two operators is the operators 
associated with the product of symbols. 

We can inverse the elliptic symbol, more precisely, let a a symbol of order m satisfying there 
exists C > such that for all G M" x R" we have \a{x,£,)\ > C(0"- Then for ah > there 

exist b a symbol of order ~m and r a symbol of order —N such that b{x, D) o a{x, D) = I + hr{x, D). 
We can localized this result. Let K a closed set of M", we assume that there exists C > such that 
for ah {x, Kx W, \a{x, 0| > C(0™ for aU x G '^o°°(M") supported in K, there exists b a symbol 
of order — m and r a symbol of order —N such that b{x,D) o a{x,D) = xi^) + hr{x,D). In both 
cases we say that 6 is a parametrix for a. 

We can also define pseudo-differential on a smooth compact manifold without boundary. We 
shall use freely the result on M" in the context of manifolds. To distinguish both cases we denote 
by Op a the operators on R" and by op a the operators on a manifold of dimension n — 1 or on 
]R"-i ^ {x G M", Xn = 0}. 

We use also spaces H^^ and the pseudo-differential calculus on these spaces. In general that 
requires introduction of the delicate notion of "transmission condition" (see Boutet de Monvel [2]), to 
avoid that we follow the Hormander's strategy (see [15, Appendix B]) adapted for the parametrices 
which are particular cases of operators satisfying the "transmission condition". We recall some results 
proved by Hormander in the context of classical H spaces. The adaptation to the H^^ spaces 
is easy and we give here only the results and some ideas of proof. Here we give the result in 
a half space R" = {x G M", a;„ > 0}. For simplicity we denote by H^^ = Hg^{R^). In the 
proof we need introduce a space i?™'*. First we say that u e iJ™'''(R") = i?™'" if = 

< oo where ^ = (CCn)- As for the space we say that u G F^'" 
where w is a distribution in ^'(M") if there exists v G H"^'^ such that u ~ i^|.t„>o ^i^d we denote 
= inf{||t;||/jm,s, whcrc v G 77™'^, such that W|2-„>o = u}. 

We can easily see that if u e H^].. then u e H'^^f and ||uj|-^o,s = , (see (14) for the 

definition of u) . 

We can extend the Theorem B.2.3 given by Hormander [15] 

■yzfm.s — rn— l,s — m — l,s+l 

u G ^ ^x„u £ and u e U 

„o — rn— 2,s —m—2,s+l — m — 2,s+2 , , 

■i^D^^ueH^^ ,Dx^ueH^^ and ueH^^ . (54) 

Of course the natural norms on these spaces are equivalent. 

We can use the Theorem B.2.9 from [15] in the following form adapted to our context. We denote 
by P = D^^^ + R{x, D') -f a{x)Dx„ a differential operator of second order. We have for all fc G M, 

— m-fe.s+fe — rji-2,s TT™'" /rr\ 

u G and Pu £ H^^ ^ u e H^^ . (55) 

Of course this is trivial if fc < 0. For fc > we can prove this by recurrence on fc. The idea is the 
following; we can write D^^u — Pu — R{x,D')u — aD^^u and for fc = 1 this formula implies that 

U^^u G Tf^gc ^ Then we can apply (54) to obtain the result in this case. The recurrence is easy. 

The previous results are useful to prove that the parametrix of a elliptic operator is a mapping 
on the H space. 

Let P a differential operator of second order, elliptic, i.e. there exists C > such that Vx G M", 
e M", |p(a;,^)| > C(^)^, and let Q a parametrix such that QP ^ Id + hK where K is of order 
—N where N > 0. For a distribution w in K", we denote by rw = W|^^>o- The action of Q on H^^ 
is given by the formula rQu which make sense if u make sense (it is the case for if m e L^(f2)). We 
have the following result. If s e [0, A^], there exists C > such that for all u G H^^, we have 

\\rQu\\j^s^2 < C\\u\\tj.^. (56) 
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Remark, here u is at least in so rQu make sense. First we prove that rQu e ' where 

k> s. It is enough to prove that for all a G N"~^, \a\ < k, D'"Qu g -ff^^^ ■ By pseudo-differential 
calculus we have D'"Q = J2\B\<\a\ Qp^'^ ^ where Qp is of order —2. We have 



\\rD'°'Qu\\jj.^2^. < ^ \\Q0D'f^u\\^sj2-u<C ^ \\D"^u\\jj.-u < C\\ u\\jjs-u,, 

l/3|<l"l " l/3|<l"l 

<C\\y\\^..,<C\\u\\^..<C\\u\\-j^.^^. 

It is well-known that we have also PQ = Id + hK where K is of order —N. Indeed there exists a 
Q such that PQ = Id + hK where K is of order —N and it is easy to prove that Q = Q + hK' where 
K' is of order — 2. Thus we have, as P is a differential operator, PrQu = rPQu = ru + hrKu = 
u + hrKu. As || Ku\\h" ^ CIIi^IIl^ < II'^IIh-' ; we obtain for s € [0, A^], \\PtQu\\w — ^ll^llff^ ■ 
Then we have rQu € H ' and PrQu G H then from (55) this implies ll^'Qujl^g^s+s < C||u||-g^s . 

We need also regularity results for rQ{j (g) Sx„=o), where 7 £ iJ|^(R"^^). First we remark 
/ /i^''ICnP''(l + h^\^n\^ + ICl^Td^n < C(f ) 2'=+2'^+7/i"ff + fc < -1/2, then by direct computation 
we have for ah 7 e II7 £)J„4„=o||H.--fc-i/2,=-^ < :^|7|h.(r»-i), if j/ < fc. For j e N, we 

have if s — fc < j, following the same computation as for computing rQu, 

C 

WQil ® D'^Jx„=o)\\h--j-''+V2.j < C\\"f ® D';J^,^=o\\Hs-,-k-i/2^j < --^\-/\h^{M^-1). 

As r(7 D^Jx^=o) = we have PrQ{-f i:>^^(52;„=o) = /irA:(7 «) -D^'„4„=o)- We deduce, 

C 

if s - k < N. Thus from (55) we obtain if s - A: < A^, 

C 

WrQil ® 7^^'^4„=o)||^=-.+3/2(jj„^ < ^|7|ff.(R„-i). (57) 

Remark 10. In the section 2.3 we apply the previous result to a local parametrix. Indeed we can 
construct the local parametrix with a global parametrix. We can extend P to have a global elliptic 
operator P such that P = P in a domain W where P is elliptic. Let Q a parametrix of P such that 
QP = Id + hK where K is an operator of order — A^. Let xi ^^'i X2 functions in 't^°° compactly 
supported in W such that X2 = 1 on the support of xi- By pseudo-differential calculus, we have 
XiQX2 ~ XiQ + f^K' where K' is an operator of order — A^ — 2. Then we have XiQP = Xi + hxiK 
and XiQP — X1QX2P — hK'P. As P = P on the support of X2 we have X1QX2P = Xi + ^AT" where 
K" is an operator of order — A^. Then Xi'3X2 is a local parametrix of P. It is easy to see that we 
can replace Q by Xi'3X2 in (56) and in (57). 



A. 2 Properties on the roots and parametrices 

We use some properties of the roots of -I- R{x, SJ) — /i and a(^^ 4- R{x, ^')) — /i. By assumption these 
polynomials have not real roots and it is easy to see that for ^' large enough the imaginary parts have 
different signs. In particular the roots are simple thus smooth and the roots are symbols of order 1. 
Actually, for instance for ^"^^ + R{x, ^') — (the proof for a(^^ + R{x, fi is similar and left to the 
reader) the roots have, for |^'| large enough, the following form ztiy^R{x, + z±{fi, 1/ y^R{x, 
where z± is a solution to z± =F iz^s/2 + i/is/2 = in a neighborhood of s = 0. This expression 
implies that the roots are symbols of order 1. 

The parametrices used, denoted by Q and Q have a particular structure we give here. The symbol 
of P is a polynomial having the following form, p2 + hpi where pj are polynomial of degree j. We 
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seek a parametrix with symbol given formally hy q = g_2 + + h?q-A + • • • , where qj are symbol 
of order j. If we denote hy qo p the asymptotic expansion of the symbol of 0'p{q) Op(p), we have, 

where in the sum we have j = 2 or 1, fc > 2, a G N" and, \a\ > 1 or j = 1. In particular we have 
k — j + \a\ > 1. We choose g_2 = 1/P2, and to cancel the terms with the same power in h we have 



J:E;i^r'?-^^.>.' (58) 



where ly > 1, k — j + \a\ = v, j = 2, 1, fc > 2. a e N", \a\ > 1 or j = 1. In particular the sum is finite 
and A: < + 1 . We claim now that 

1-:^ -^U^ for iy>2, 
P2 

where S'^ is a polynomial of degree fi. 

Clearly this is true for = 2. We verify that for k < v + 1, dfq-k = 2fc~3+iol 1 where Sn is a 

« P2 

polynomial of degree /x. The parameters satisfy j < 2, k — j + \a\ = v and fc < + 1, then the power 
of p2 in (58) is 2fc - 2 + |a| = 1/ + fc + j - 2 < 2z^ + j - 1 < 2i^ + 1 = 2(:^ + 2) - 3. The degree of the 
numerator is 3fc - 6 + |q;| + 2j = + 2fc - 6 + 2j < - 4 + 2j <2iv = 2i{v + 2) - 6. This gives the 
claim. 

We need to compute for 7(x'), 



Q ( -7® i:'^„4„=c 



1 



, e"'«'/\~(x',^')^(^7/j)de', (59) 

where formally q(a;',^') = (^-^ /r ^""^"^''^(^^i OCn'^Cn) |^ _q- It is not clear that q is well defined 
in general but in the following lemma we prove this is true if g is a rational function, and in this 
case (59) make sense. 

Lemma A.l. Let v G N*, let Si,{x,^) a polynomial of order v with respect ^,-1 o,nd we assume that 
the coefficient of ^ij^ is a symbol in ^' of order ly — j ■ Let p a polynomial of degree d in and a symbol 
of order d. We assume that p{x,^) = + '^j^q S,^iO-d-j{x,£,') where ad-j are polynomials of order 
d — j . Moreover we assume there exists S > such that Vx e M", G M", \p{x, ^)| > S{£^)'^ . Then 



Lg„ 'S't.(a:,g) 



\x„=0 



is a symbol of order v — d + I. 

Proof. The integral /r e'"i?"5" converges for a;„ > 0. For (a;,C') fixed, we can change 

the integration contour by T = [-£»'(^')D(f )] U {z G C, \z\ = D{^'), Imz > 0}, where D will 
be chosen later. Indeed the integral does not depend of D if D large enough and the integral on 
{z G C, \z\ = L){£^'), Imz > 0} goes to if D goes to +00. Now we integrate on a compact set and 
we can take the limit when Xn goes to 0+. We have the following quantity to control. 

On r we have |S'^(0, x', ^1 < C{Cy. For ^„ G {;2 G C, \z\ = D{C), Imz > 0} we have 
\p{0,x',O\ > \Cnf - EjZo \&.\'\ad-j{0,x',^')\ > D^iCyil - dD~^C) > (O^^VS, where we have 
\ad-j{0,x' < C{^')'^^^ and chosen D such that D > max(l,2(iC). Then by assumption for all 
^„ G F we have |p(0,x',C)l > As the length of 7 is less than K{C) we obtain A < K' {£^'Y-'^+^ . 

We can obtain the estimates on the derivative by the same way because we can derive A and we 
obtain the same type of quantities to estimate. ■ 
Now we compute the boundary symbol obtained in (21) and (24). 
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Lemma A. 2. Let fc = 0, 1 and Impi > 0, Imp2 < 0, we have 

Proof. As in the proof of Lemma A.l, we can integrate on F and this integral is equal to 2i7r 
times the residu at pi. It is easy to see that the residu is j^zr^- ■ 

Lemma A. 3. Let fc = 0, 1 and ImAi > 0, ImA2 < 0, Impi > 0, Imp2 < 0, we have 
e'^^^-TT TTTT M7 T77 = 2z7rAfe, 



(Cn - Ai)(^„ - \2){in - - P2) J |^„^o 

l/fc = 



where 

/52 - Pi + A2 - Ai 



A, = 



(Ai - A2)(Ai - P2){pi - A2)(pi - P2) 

A2P2 - AijOi 

(Ai - A2)(Ai - P2)(pi - A2)(pi - P2) 



•*/fc = l. 



Proof. Clearly both sides of the equality are continuous with respect (Ai,pi) then it is sufhcient 
to prove the case Ai ^ pi- 

As in the proof of Lemma A.l, we can integrate on F and the result is 2z7r times the sum of the 
residues in half plane Im z > 0. We obtain 



-^1 , Pi 



k 



(Ai - A2)(Ai - pi)(Ai - P2) [pi - Ai)(pi - A2)(yOi - P2) 

A^(pi - A2)(pi - P2) - P?(Ai - A2)(Ai - P2) 



(Ai - A2)(Ai - pi)(Ai - P2)(pi - A2)(pi - P2) ' 



Clearly the numerator is null if pi = Ai. By a straightforward computation, if fc = the numerator 
is (Ai — Pi)(p2 ^ Pi + A2 — Ai). If fc = 1 the numerator is (Ai — pi)(A2P2 — Aipi). This gives the 
Lemma. ■ 
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